Multimodal Analysis Research Articles

Multimodal detection and analysis of microplastics in human thrombi from multiple anatomically distinct sites

Microplastic (MP) pollution has emerged as a significant environmental concern worldwide. While extensive research has focused on their presence in marine organisms and ecosystems, their potential impact on human health, particularly on the circulatory system, remains understudied. This project aimed to identify and quantify the mass concentrations, polymer types, and physical properties of MPs in human thrombi surgically retrieved from both arterial and venous systems at three anatomically distinct sites, namely, cerebral arteries in the brain, coronary arteries in the heart, and deep veins in the lower extremities. Furthermore, this study aimed to investigate the potential association between the levels of MPs and disease severity. Thrombus samples were collected from 30 patients who underwent thrombectomy procedures due to ischaemic stroke (IS), myocardial infarction (MI), or deep vein thrombosis (DVT). Pyrolysis-gas chromatography mass spectrometry (Py-GC/MS) was employed to identify and quantify the mass concentrations of the MPs. Laser direct infrared (LDIR) spectroscopy and scanning electron microscopy (SEM) were used to analyse the physical properties of the MPs. Demographic and clinical information were also examined. A rigorous quality control system was used to eliminate potential environmental contamination. MPs were detected by Py-GC/MS in 80% (24/30) of the thrombi obtained from patients with IS, MI, or DVT, with median concentrations of 61.75μg/g, 141.80μg/g, and 69.62μg/g, respectively. Among the 10 target types of MP polymers, polyamide 66 (PA66), polyvinyl chloride (PVC), and polyethylene (PE) were identified. Further analyses suggested that higher concentrations of MPs may be associated with greater disease severity (adjusted β=7.72, 95% CI: 2.01-13.43, p<0.05). The level of D-dimer in the MP-detected group was significantly higher than that in the MP-undetected group (8.3±1.5μg/L vs 6.6±0.5μg/L, p<0.001). Additionally, LDIR analysis showed that PE was dominant among the 15 types of identified MPs, accounting for 53.6% of all MPs, with a mean diameter of 35.6μm. The shapes of the polymers detected using LDIR and SEM were found to be heterogeneous. This study presents both qualitative and quantitative evidence of the presence of MPs, and their mass concentrations, polymer types, and physical properties in thrombotic diseases through the use of multimodal detection methods. Higher concentrations of MPs may be associated with increased disease severity. Future research with a larger sample size is urgently needed to identify the sources of exposure and validate the observed trends in the study. This study was funded by the SUMC Scientific Research Initiation Grant (SRIG, No. 009-510858038), Postdoctoral Research Initiation Grant (No. 202205230031-3), and the 2020 Li Ka Shing Foundation Cross-Disciplinary Research Grant (No. 2020LKSFG02C).

Single and Multi-modal Analysis for Parkinson’s Disease to Detect Its Underlying Factors

AbstractParkinson’s disease (PD) is a neurological condition characterized by the disruption of both motor and non-motor functions. Given the absence of a definitive diagnostic method, it is crucial to uncover its root causes. Consequently, individuals displaying symptoms of Parkinson’s disease can promptly receive treatment and comprehensive care. To address this, our study aims to develop an AI-powered system capable of detecting Parkinson’s disease and subsequently evaluating the primary factors influencing its development. We collected 12 distinct datasets from the well-known PPMI database, covering various medical assessments such as motor abilities, olfaction, cognition, sleep patterns, and depressive symptoms. Subsequently, we refined this raw data using advanced search techniques to tailor it to our model’s requirements. Moreover, we introduced a novel labeling approach known as the majority voting algorithm. Following data preparation, we conducted Single and Multi-Modality analyses, focusing on single-treatment approaches and integrating multiple treatments for a comprehensive therapeutic strategy. To analyze these both, we employed five distinct Machine Learning algorithms. Notably, the Support Vector Machine (linear) emerged as the top performer, reaching an accuracy of 100% in both single and multimodality analysis. Furthermore, we employed four tree-based models for feature selection, with the Gradient Boosted Decision Tree excels in identifying the most significant features. Finally, we employed an Artificial Neural Network utilizing these key features, achieving the highest accuracy of 91.41%.

Stance and food activism on Arabic Twitter (X): a multimodal analysis

Abstract Expanding on classic theories of stancetaking and theorizing digital activism as nexus analysis, I examine how Omani Arab citizens used discourse and images to canalize a failed food boycott campaign on Twitter (X) into a united negative stance against alleged corruption. I highlight the role of multimodality, impoliteness, indirectness, intertextual references, participation framework and Arabic cultural practices in creating a multi-layered, accreted stance with manifold implicit functions (e.g., defining Omani identity, lamenting and signalling dissent). I argue for the need to widen the scope and level of analysis of stance acts to include interplays between texts, emoji, actions and images. I also highlight the role that stance can perform in expressing dissent and managing cultural face in the understudied Arabic context. I, therefore, demonstrate the multidimensionality of stance acts made visible by social media affordances and user creativity.

How emotional cues affect the financing performance in rewarded crowdfunding? - an insight into multimodal data analysis

How emotional cues affect the financing performance in rewarded crowdfunding? - an insight into multimodal data analysis

Text-image semantic relevance identification for aspect-based multimodal sentiment analysis

Aspect-based multimodal sentiment analysis (ABMSA) is an emerging task in the research of multimodal sentiment analysis, which aims to identify the sentiment of each aspect mentioned in multimodal sample. Although recent research on ABMSA has achieved some success, most existing models only adopt attention mechanism to interact aspect with text and image respectively and obtain sentiment output through multimodal concatenation, they often neglect to consider that some samples may not have semantic relevance between text and image. In this article, we propose a Text-Image Semantic Relevance Identification (TISRI) model for ABMSA to address the problem. Specifically, we introduce a multimodal feature relevance identification module to calculate the semantic similarity between text and image, and then construct an image gate to dynamically control the input image information. On this basis, an image auxiliary information is provided to enhance the semantic expression ability of visual feature representation to generate more intuitive image representation. Furthermore, we employ attention mechanism during multimodal feature fusion to obtain the text-aware image representation through text-image interaction to prevent irrelevant image information interfering our model. Experiments demonstrate that TISRI achieves competitive results on two ABMSA Twitter datasets, and then validate the effectiveness of our methods.

A Two-Stage Stacked Transformer Framework for Multimodal Sentiment Analysis

A Two-Stage Stacked Transformer Framework for Multimodal Sentiment Analysis

Transformer-based adaptive contrastive learning for multimodal sentiment analysis

Transformer-based adaptive contrastive learning for multimodal sentiment analysis

All eyes on the signal? - Mapping cohesive discourse structures with eye-tracking data of explanation videos

In this paper, we consider the issue of how the fine-grained multimodal design of educational explanation videos, such as those widely available on YouTube and other platforms, may be made accessible to empirical studies of reception and effectiveness. This is necessary because previous research has often led to conflicting conclusions concerning the roles of particular design elements. We argue that this may largely be due to insufficient characterizations of multimodal design itself. To achieve tighter control of this potential source of variation, we present a multimodal descriptive annotation framework drawing on multimodal (cohesive) film discourse analysis. This framework is seen as a critical first step toward being able to highlight just those differences in design that have functional consequences. For such consequences to accrue, however, viewers need to attend differently to corresponding design differences. The goal of the current paper, therefore, is to use eye-tracking techniques to explore the extent to which discourse structures revealed by our analytic framework relate to recipients' attention allocation. We hypothesize that any potentially emerging anomalies in regards to discourse organization, such as instances of unsuccessful cohesion signaling, may have correlations in the behavioral data. We report our current state of development for performing this kind of multimodal cohesion analysis and some of the unresolved challenges raised when considering how such analyses may be related to performance data.

Modal acoustic emission-based circumferential crack feature extractions for pipeline welds with L-shaped flexible sensor array

ABSTRACT The collection and analysis of acoustic emission (AE) from weld circumferential cracks are crucial for ensuring pipeline safety. However, the unclear propagation characteristics and mode features of AE present challenges in array design and mode identification. In this manuscript, a novel mode feature extraction method for circumferential crack AE signals is proposed. This method involves three primary steps: firstly, a L-shaped flexible sensor array is designed to capture raw AE signals from both axial and circumferential directions. Next, the collected signals are filtered and evaluated to extract effective multi-mode components through the Kalman filtering and recursive plot (RP). Finally, the time–frequency features and mode types are extracted through combining the Hilbert-Huang transform (HHT) and dispersion curves. Results indicate that during the crack growths, both axial and circumferential direction AE signals contain multi-mode components, specifically L(0,1), F(1,1) and F(1,2) modes, with durations spanning 30–500 μs. Additionally, the axial modes predominantly occur within 200–300 kHz range, whereas the circumferential modes span both low and high frequency bands, specifically 40–50 kHz and 200–300 kHz, respectively. Modes across different frequencies indicate distinct structural behaviours, including crack growths and plastic deformations. The proposed method provides attempts for the pipe weld AE monitoring and multi-mode analysis.

Face-to-Face &amp; digital: Die interaktive und multimodale Herstellung digitalisierter Handlungsräume in Kopräsenz

Abstract In this contribution we analyse how mobile device users in face-to-face communication jointly negotiate the boundaries and action spaces between digital and non-digital, shared and individual, public and private. Instead of conceptualising digital and face-to-face, i. e., non-digital, communication as separate, more recent research emphasises that social practices relying on mobile devices increasingly connect physical and virtual communicative spaces. Using multimodal conversation analysis, we investigate the situated use of mobile devices and media in social interaction. Excerpts from videotaped everyday conversations illustrate how participants frame their smartphone use in the presence of others, such as when looking at digital pictures, or when recording voice messages. A detailed analysis of verbal and embodied conduct shows how participants negotiate and interpret the connection or separation of digital and non-digital activities and possible forms of participation within these. (Digital) publicness or privacy are therefore to be understood as an interactive accomplishment.

Multimodal imaging analysis in silver fir reveals coordination in cellulose and lignin deposition.

Despite lignin being a key component of wood, the dynamics of tracheid lignification are generally overlooked in xylogenesis studies, which hampers our understanding of environmental drivers and blurs the interpretation of isotopic and anatomical signals stored in tree rings. Here, we analyzed cell wall formation in silver fir (Abies alba Mill.) tracheids to determine if cell wall lignification lags behind secondary wall deposition. For this purpose, we applied a multimodal imaging approach combining transmitted light microscopy (TLM), confocal laser scanning microscopy (CLSM), and confocal Raman microspectroscopy (RMS) on anatomical sections of wood microcores collected in northeast France on 11 dates during the 2010 growing season. Wood autofluorescence after laser excitation at 405 nm and 488 nm associated with the RMS scattering of lignin and cellulose, respectively, which allowed identification of lignifying cells (cells showing lignified and non-lignified wall fractions at the same time) in CLSM images. The number of lignifying cells in CLSM images mirrored the number of wall-thickening birefringent cells in polarized TLM images, revealing highly synchronized kinetics for wall thickening and lignification (similar timings and durations at the cell level). CLSM images and RMS chemical maps revealed a substantial incorporation of lignin into the wall at early stages of secondary wall deposition. Our results show that most of the cellulose and lignin contained in the cell wall undergo concurrent periods of deposition. This suggests a strong synchronization between cellulose and lignin-related features in conifer tree-ring records, as they originated over highly overlapped time frames.

Assessment of the FRET-based Teen sensor to monitor ERK activation changes preceding morphological defects in a RASopathy zebrafish model and phenotypic rescue by MEK inhibitor

BackgroundRASopathies are genetic syndromes affecting development and having variable cancer predisposition. These disorders are clinically related and are caused by germline mutations affecting key players and regulators of the RAS-MAPK signaling pathway generally leading to an upregulated ERK activity. Gain-of-function (GOF) mutations in PTPN11, encoding SHP2, a cytosolic protein tyrosine phosphatase positively controlling RAS function, underlie approximately 50% of Noonan syndromes (NS), the most common RASopathy. A different class of these activating mutations occurs as somatic events in childhood leukemias.MethodHere, we evaluated the application of a FRET-based zebrafish ERK reporter, Teen, and used quantitative FRET protocols to monitor non-physiological RASopathy-associated changes in ERK activation. In a multi-level experimental workflow, we tested the suitability of the Teen reporter to detect pan-embryo ERK activity correlates of morphometric alterations driven by the NS-causing Shp2D61G allele.ResultsSpectral unmixing- and acceptor photobleaching (AB)-FRET analyses captured pathological ERK activity preceding the manifestation of quantifiable body axes defects, a morphological pillar used to test the strength of SHP2 GoF mutations. Last, the work shows that by multi-modal FRET analysis, we can quantitatively trace back the modulation of ERK phosphorylation obtained by low-dose MEK inhibitor treatment to early development, before the onset of morphological defects.ConclusionThis work proves the usefulness of FRET imaging protocols on both live and fixed Teen ERK reporter fish to readily monitor and quantify pharmacologically- and genetically-induced ERK activity modulations in early embryos, representing a useful tool in pre-clinical applications targeting RAS-MAPK signaling.

Representing slavery in visual art: a multimodal approach

Abstract This article explores some issues related to the trauma of slavery, colonialism and identity-formation processes through a multimodal analysis, which is particularly useful in studying the manifold meanings and relationships emerging from the forms of cultural and social representation such as visual art. A series of images, chosen within Ellen Gallagher’s art exhibition “AXME”, held at Tate Gallery in London, is investigated to deal with the discursive constructions of gendered and racial identities. Employing Kress and van Leeuwen’s model (2021 [1996]. Reading images: The grammar of visual design. London: Routledge), this article attempts to identify the signs related to categories such as gender and race, as well as the persistence of stereotypical representations in our contemporary societies.

Photography and Destination Image: An Analysis of China Eastern Airline Facebook Photographs

ABSTRACT The main purpose of this study is to explore the projected destination image represented by photographs emerging in social media from a potential DMO member. The official page of China Eastern Airlines on Facebook (@CEAir) was selected as a case study, in which a total of 40 photographs were collected and classified into five groups according to their themes. To identify the destination image of China, five groups of photographs were analyzed by using visual analysis, semiotic analysis, and multimodal discourse analysis. The main findings show that Chinese food, architecture, natural landscape, people, and ethnic minorities are common signs engaging in constructing the destination image of China. These photographs mainly adopted wide shots and brilliant color tones, which convey to overseas tourists that China is a great power with a long history, full of charm, and a contributor to global development. More importantly, not only landscape photography but also still life photography, candid photography, and travel photography are all considered to be modes of representation of the destination image. Different genres of photography employ Chinese culture as a special gaze to shape the destination image.

Whose questions? Ventriloquation in entrepreneurial podcasts

While interaction is a signature feature of podcasts, new research on selected entrepreneurial podcasts has shown how they are de-dramatized as part of the leadership branding project, indexing the genre’s transition from a homespun to a corporate medium and echoing the notion of media re-colonization. This article reports on a case study of the use of questions in a single entrepreneur’s six-episode podcast series. We set out to identify and analyse what we see as ‘re-dramatization’ strategies, creating a sense of interaction and heteroglossia in what is essentially a monologue. Next, triangulating with insights from multimodal discourse analysis and linguistic ethnography, our analysis points to ventriloquation. Through questions that are presented as reported (i.e. referring back to questions that were raised on some earlier occasion), the entrepreneur makes himself or others say something, staging a specific past situation (real or imagined) where questions were asked and thus creating an enhanced sense of author-ity. As for questions that are presented as asked at the moment of speaking (we call them direct questions), the results of our inquiry indicate that the entrepreneur is ventriloquating either what he believes the listener might want to ask him or what he himself was previously asked by the podcast producer. We suggest that the entrepreneur is accommodating more actively to his listenership than it seems at first sight and conclude by reflecting on how this sheds new light on the entrepreneurial podcast as a tool of leadership communication as part of the wider digital mediascape, on the interactive use of questions in general and on the potential of our multi-method approach.

Effects of Wind-induced Static Angle of Attack on Flutter Performance of Long-span Bridges using 2D Bimodal and 3D Multimodal Analysis

Flutter performance of long span bridges is directly affected by winds featured with large angles of attack (AoAs) in typhon climates and mountainous areas. Flutter critical wind speed and flutter derivatives (FDs) of a streamlined closed-box girder under various initial AoAs are obtained by free and forced vibration wind tunnel tests of a section model, based on which the two-dimensional(2D) and three-dimensional (3D) flutter performance and evolution mechanisms are further investigated. The results indicate that FDs identified at small vibration amplitudes through different forced vibration control methods, i.e., fixing wind speed or vibration frequency, in wind axis or body axis are almost identical. The flutter performance of the closed-box girder is susceptible to the AoA. The flutter critical wind speed changes significantly with the initial AoA due to the nonlinear variation of some typical FDs. The sign of A2⁎ shifts from negative to positive and the reduced wind speed associated with the sign change becomes lower as the initial AoA increases, which worsens the flutter performance. The flutter critical wind speed would significantly decrease because of the consideration of the static additional AoA in the range of positive initial AoA. In addition, the participation of multi modes and different longitudinal distribution of static additional AoA are taken into consideration in 3D numerical analysis, which have little influence on the flutter critical wind speed of long span bridges between the 2D and 3D numerical analysis.

Abstract LB311: Multimodal spatial profiling reveals the emergence of an immune suppressive microenvironment at the initial stages of high-grade serous ovarian cancer development

Abstract Introduction: Despite the clear genetic evidence linking serous tubal intraepithelial carcinoma (STIC) and high-grade serous ovarian carcinoma (HGSOC), the specific conditions and events that promote the progression of STIC lesions into invasive disease remain poorly understood. Method: As a critical initial step, we have assembled a cohort of incidental p53 signatures, STIC lesions, and STIC with concurrent HGSOC. We have performed extensive multi-modal analysis using multiplexed tissue imaging and spatial transcriptomics that identify features of the immune system that play a vital role in the early steps of HGSOC development. We have processed 43 specimens using highly multiplexed tissue imaging at single-cell resolution (cyclic immunofluorescence, CyCIF), and 35 specimens for micro-regional spatial transcriptomics using the GeoMx (Whole Transcriptome, Nanostring) on over 450 pathologist-annotated regions of interest. Results: Our data suggests an immune-cold environment and T-cell dysfunction in STIC lesions, including incidental STIC. One of the significant immune populations identified was CD103+ tissue-resident memory T cells (TRM). In incidental p53 signatures, activation of TRM was rare, similar to FT, which may indicate the absence of sensing “tumor antigen” by these TRM. CyCIF analysis also revealed that most incidental STIC lesions (7/9 cases) overexpress major histocompatibility complex (MHC) class I compared to the normal epithelium, especially both HLA complex, HLA-A and HLA-E. Most STICs showed extensive intra-lesion heterogeneity, with some STICs remaining HLA-A and HLA-E negative. Incidental p53 signatures, on the other hand, were mostly HLA-A and HLA-E negative and, when positive, had only a few cells expressing HLA. We hypothesized that there might be a natural selection of HLA-E-positive clones as STICs progress to HGSOC. Consistent with this, HGSOC showed further overexpression of HLA-E. However, HLA-E heterogeneity was still observed in the invasive tumor, with both positive and negative clones co-existing. Geomx data suggested that the interferon signaling pathway is upregulated in the epithelial of HLA-E positive STIC and cancer clones compared to HLA-E negative clones. In turn, both HLA-A and HLA-E might be overexpressed in the epithelial of HLA-E-positive STICs. Overall, we showed that the response in interferon (IFN) α and γ, NF-KB, and IL-6-induced STAT-3 pathways were upregulated in both STIC lesions and carcinoma. The role of these pathways, especially STAT-3 pathway, has been shown in other aneuploid cancers in promoting immune escape, cell proliferation and migration, chemoresistance and inhibiting apoptosis. We have shown the co-localization of cGAS and BAF, a marker for micronuclei rupture, by super-resolution 3D imaging to confirm one of the mechanisms of IFN activation. Conclusion: Taken together, these data may indicate chromosomal instability is one of the mechanisms that is driving the IFN-signaling pathway and, hence, a potential selective advantage for HLA-E-positive clones in tumorigenesis, leading to inhibiting NK cell surveillance followed by reducing T cell infiltration. Citation Format: Tanjina Kader, Jia-Ren Lin, Shannon Coy, Clemens Hug, Yu-An Chen, Roxanne J. Pelletier, Mariana Leon, John Lee, Yi-Lin Xu, Clarence Yapp, Natalie Shih, Gabriel Mingo, Euihye Jung, Srishti Rathore, Judith Agudo, Charles Drescher, Peter K. Sorger, Ronny Drapkin, Sandro Santagata. Multimodal spatial profiling reveals the emergence of an immune suppressive microenvironment at the initial stages of high-grade serous ovarian cancer development [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 2 (Late-Breaking, Clinical Trial, and Invited Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(7_Suppl):Abstract nr LB311.

Abstract LB102: Triaging oral premalignant lesions: Plasma ultrashort cell-free DNA-based liquid biopsy approach

Abstract Background: Rapid advancements in cell-free DNA (cfDNA) based liquid biopsy (LB), has opened avenues of critical clinical importance. Recent discovery of a novel class of cell-free DNA 40-70 bps in length, ultrashort cfDNA (uscfDNA) is demonstrating immense potential as a cancer early detection biomarker. Globally, oral cavity squamous cell carcinomas (OCSCC) make up a significant proportion of head and neck squamous cell cancers. OCSCC commonly arises after the occurrence of oral premalignant lesions (OPLs), making it an ideal target for plasma-based LB screening initiatives. Methods: Current work focuses on preferential extraction of low molecular weight nucleic acids and single-stranded library preparation and sequencing, Broad Range cell-free DNA- Sequencing (BRcfDNA-Seq), to obtain uscf- &amp; 167 bps mononucleosomal cfDNA (mncfDNA), for disease detection. Sequencing data was bioinformatically processed: paired-end reads merged (BBmerge), aligned with BWA-mem, deduplicated using SRSLY UMI, and blacklisted portions removed. This was followed by analysis for non-mutational features like fragmentomics, occurrence of G-Quad structures, 4-mer Motif sequences at the end of cfDNA sequences, functional genomic element peak formation by uscfDNA. These features have been recently described non-mutation metrics which have demonstrated evidence of clinical relevance. With non-mutational multi-modal analysis of uscfDNA, obtained using BRcfDNA-Seq, we were able to categorize oral premalignant disorders into two cohorts: one which progressed into OCSCC (PG-OPL) (n=19) &amp; one which did not (nPG-OPL) (n=19). Results: Bioinformatic processing for fragmentomics, of the relative proportion of 40-53bps fragments to 54-70bps fragments in the uscfDNA population demonstrated a significant difference across 1700 chromosomal bins between PG &amp; nPG-OPL (multiple t test, FDR 10%, p value = 0.044, AUROC = 0.7258). The occurrence of G-quadruplex structures in uscfDNA was also significantly different between the two groups (student t test, p= 0.02, AUROC= 0.6953). Examination of the end-motifs profiles of uscfDNA between PG-OPL &amp; nPG-OPL samples showed the randomness of the motifs as revealed by the Shannon entropy score is significant between the two groups (p = 0.046, AUROC = 0.6482). We further evaluated individual motifs that could discriminate between the two groups and identified CCCC, CCCT, TGGG, etc. are highly discriminatory. uscfDNA contained a substantially higher proportion of functional genomic element sequences in the PG-OPL group than the nPG-OPL group like exon (p = 0.00067, AUROC = 0.7147) &amp; 5’ UTR (p= 0.000006, AUROC= 0.7673) are different between the groups. Conclusion: Results demonstrated clinical relevance of BRcfDNA-Seq for plasma-derived cfDNA in assessment of malignancy progression of OPLs. These non-mutational multi-modal features of cfDNA maybe indicative of local and systemic factors associated with disease affecting cfDNA. Citation Format: Neeti Swarup, Jordan C. Cheng, Irene Choi, Mohammad Arshad Aziz, Kelly Y. Liu, Fang Wei, Feng Li, Yong Kim, Catherine Poh, David T. Wong. Triaging oral premalignant lesions: Plasma ultrashort cell-free DNA-based liquid biopsy approach [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 2 (Late-Breaking, Clinical Trial, and Invited Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(7_Suppl):Abstract nr LB102.

Neoliberal, trouble-free worlds for an aspirational middle-class in Chinese EFL publications: A multimodal critical discourse analysis

For most English language learning engagements in expanding circle countries, textbooks play a key role in defining a syllabus and corralling students toward some measure of completion in a course of study. While scholars have shown how English is broadly seen as a prestigious skill, connected with upward socio-economic mobility, less work has been done on the discourses and ideologies carried by the textbooks, even though this is a billion-dollar-a-year industry with titles selling in the millions. Research clearly raises the need for further investigations of the discourses and world views that they carry as they appear in different territories. In this paper, using Multimodal Critical Discourse Analysis, we analyze a corpus of texts books produced domestically within China. In China, English language learning has become a fundamental part of the aspirations of the emerging middle classes, who also tend to consume Western-style products and increasingly take on more individualistic Western-style values. Analysis shows that these textbooks carry language learning activities which combine a gloss of signifiers of more local, cultural ideas embedded in a privileged world inhabited by an aspirational, international middle-class.

ЗАСТОСУВАННЯ ГЛИБОКИХ ШТУЧНИХ НЕЙРОННИХ МЕРЕЖ ДЛЯ КЛАСИФІКАЦІЇ МУЛЬТИМОДАЛЬНИХ ДАНИХ

Multimodal data analysis is gaining attention in recent research. Pu Liang et al. (2023) provide a comprehensive overview on multimodal machine learning, highlighting its founda-tions, challenges and achievements in recent years. More problem-oriented works propose new methods and applications for multimodal ML, such a Ngiam et al. (2011) propose to use joint audio and video data to improve speech recognition accuracy; Sun, Wand and Li (2018) describe application of multimodal classification for breast cancer prognosis prediction; Mao et al. (2014) propose an architecture of multimodal recurrent network to generate text de-scription of images and so on. However, such works usually focus on the task itself and meth-ods therein, and not on integrating multimodal data processing into other software systems. The goal of this research is to propose a way to conduct multimodal data processing, specifically as a part of a digital twin systems, thus efficiency and near-real-time operation are required. The paper presents an approach to conduct parallel multimodal data classification, adapting to available computing power. The method is modular and scalable and intended for in digital twin application as a part of analysis and modeling tools. Later, the detailed example of such a software module is discussed. It uses multimodal data from open datasets to detect and classify the behavior of pets using deep learning mod-els. Videos are processed using two artificial neural networks: YOLOv3 object detection net-work to process individual frames of the video and a relatively simple convolutional network to classify sounds based on their frequency spectra. Constructed module uses a producer-consumer parallel processing pattern and allows processing 5 frames per second of a video on available hardware, which can be sufficiently improved by using GPU acceleration or more paralleled processing threads.